Visual Attention, Speaking Activity, and Group Conversational Analysis in Multi-Sensor Environments

Among the many possibilities of automation enabled by multi-sensor environments several of which are discussed in this Handbook one particularly relevant is the analysis of social interaction in the workplace, and more specifically, of conversational group interaction. Group conversations are ubiquitous, and represent a fundamental means through which ideas are discussed, progress is reported, and knowledge is created and disseminated. It is well known that spoken words represent a fundamental communication channel. However, it is also known from research in social and cognitive science and experienced by all of us everyday that nonverbal communication is also a key aspect in social interaction, through which a wealth of personal information, ranging from instantaneous internal states to personality traits, gets expressed, interpreted, and weaved, along with speech, into the fabric of daily interaction with peers and colleagues (Knapp and Hall, 2005). In this view, although it is clear that further work on speech and language processing is needed as important modules of ”smart” environments, the computational analysis of nonverbal communication is also a relevant domain and has received increasing attention (Gatica-Perez, 2008). The coordination of speaking activity (including turn-taking and prosody) and gaze (i.e., visual attention) are key aspects of nonverbal communication. Gaze is an important nonverbal cue with functions such as establishing relationships (through mutual gaze), expressing intimacy, and exercising social control. Furthermore, the role of gaze as a cue to regulate the course of interaction, via turn holding, taking,